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Abstract 


Without a pilot onboard an aircraft, a Detect-and-Avoid (DAA) system, in conjunction 
with surveillance sensors, must be used to provide the remotely-located Pilot-in-Command 
sufficient situational awareness in order to keep the Unmanned Aircraft (UA) safely separated 
from other aircraft. To facilitate safe operations of UA within the U.S.’ National Airspace System, 
the uncertainty associated with surveillance sensors must be accounted for. An approach to 


mitigating the impact of sensor uncertainty on achievable separation has been developed to 
support technical requirements for DAA systems. 


1 Introduction 


With an increasing interest from industry and civil entities in flying Unmanned Aircraft 
Systems (UAS) within the United States’ National Airspace System (NAS), operational analyses 
are occurring to enable access while maintaining the NAS’ high level of safety and operational 
efficiency. Without a human pilot on-board to see and avoid other aircraft, UAS pilots in remote 
ground control stations must rely on sensors and Detect-and-Avoid (DAA) systems to detect 
intruder aircraft and provide suggestive guidance to maintain a safe separation threshold. 
Further, human pilots must remain well clear from other aircraft using a subjective separation 
threshold based on the pilot’s judgement and experience. DAA systems need a quantitative 
separation threshold for use in guidance and alerting calculations. Using a quantitative definition 
of a separation threshold, DAA systems can provide suggestive guidance and situational 
awareness via an alerting scheme to the UAS pilot to assist in safe flight operations. 

RTCA Special Committee 228 (SC-228), a federal advisory committee consisting of 
government and industry stakeholders, has drafted minimum operational performance 
standards (MOPS) to provide technical requirements for DAA systems. Part of the MOPS 
development effort was defining a quantitative separation threshold, termed DAA Well Clear 
(DWC), which defines a spatial- and temporal-based separation threshold for use in DAA 
systems. Also, an alerting scheme was defined with respect to the DWC definition. Much of the 
MOPS, including the DWC and alerting scheme definitions, were created using perfect 
surveillance data. Analysis introducing sensor uncertainty typical of real-life flight operations to 
the defined MOPS requirements must be completed. 

To provide timely alerts and maneuver guidance to the UAS operator, the DAA system 
must account for the imprecision of sensed intruder position and velocity. This paper presents 
an approach to account for sensor errors, how the approach was tuned, and an assessment of 
its effectiveness . The methodology is referred to as the Sensor Uncertainty Mitigation (SUM) 
approach. The SUM approach uses the horizontal and vertical position and velocity standard 
deviations provided by the tracker to augment the sensed position of each intruder with 
additional ‘phantom’ intruders. The intruder and phantom intruders are all passed to the DAA 
algorithm resulting in a block of intruders that span a sigma-multiple of the possible intruder 
locations and velocities arrayed around the sensed position and velocity. 


2 Background 


To assist the Federal Aviation Administration (FAA) in integrating UAS in to routine 
operations of the NAS, civil and industry stakeholders have developed minimum technical 
requirements for DAA systems through SC-228. The DAA system is a key component of a VAS 
responsible for providing the pilot in command situational awareness through multiple sensors 
and a defined alerting scheme. In order to develop the Phase 1 MOPS for DAA systems, SC- 
228 defined operational assumptions for use of UAS not authorized for operation through 14 
Code of Federal Regulations (CFR), Part 107 (i.e., small UAS). The MOPS restricts UAS flight 
Operations to Instrument Flight Rules (IFR) traversing through Class D, E, and G airspace up to 
Class A airspace which begins above 18,000 ft MSL. Within Class A airspace, it is assumed 
that air traffic services will provide active separation service; thus, is out of scope for this MOPS. 
Terminal airspace is not considered. 


2.1 Quantitative Definition of Well Clear 


Per 14 CFR 91.113 [1], “vigilance shall be maintained by each person operating an 
aircraft so as to see and avoid other aircraft. When a rule of this section gives another aircraft 
the right-of-way, the pilot shall give way to that aircraft and may not pass over, under, or ahead 
of it unless well clear’. Thus, utilizing an onboard pilot’s ability to see and avoid traffic, an 


aircraft must remain ‘well clear’ from other vehicles at all times. With a pilot onboard, the 
definition of well clear is qualitative, based on the ability, judgment, and preferences of the pilot 
in command. Without an onboard pilot, a quantitative definition of well clear is required to 
establish separation requirements for unmanned aircraft DAA systems. A quantitative definition 
of well clear provides a repeatable target for which a DAA system may support safe separation 
from other aircraft by providing information to the pilot or to automation. The DAA Well Clear 
volume is intended to include interoperability principles with respect to Air Traffic Control (ATC) 
operations and current collision avoidance systems such as the Traffic alert and Collision 
Avoidance System (TCAS). To ensure interoperability with the current NAS, values used to 
define the well-clear volume must be large enough to avoid issuance of TCAS corrective 
resolution advisories but not so large as to interfere with ATC separation services [2]. This 
definition has been discussed and refined through a process involving NASA, DoD, FAA, and 
SC-228. 

The proposed definition of a DAA Well Clear volume is represented using the following 
inequality: 


Equation 1: DAA Well Clear Definition 
[0 < Tmoa S Tmoqg -and. HMD < HMD*].and.[0 <1, <tj.or.—h* <dh<h*] 


where HMD is the kinematic projection of horizontal miss distance in feet, t,, is the vertical tau 
or time to co-altitude in seconds, and dh is the vertical separation in feet between the two 
aircraft involved in the encounter. The value Of Tq iS inherited from TCAS [3] and given in 
seconds by 


Equation 2: DAA Well Clear Definition 


DMOD?2 —r? 
Tmod = 
where TCAS defines DMOD as a horizontal distance threshold with varying values depending 
on the ownship’s altitude and r = range and 7 = range rate between the two aircraft. For the 
purposes of the DAA Well Clear definition, the HMD* is used in place of DMOD. In Equations 1 
and 2, the lack of a 


Table 1. Proposed DAA Well-Clear definition. superscript denotes the 
instantaneous value at any 
Parameter Symbol | __ Units Value given time during the 
, ' encounter. The superscript 
Vertical Displacement h eet 450 * denotes the value given 
Modified Tau Tod seconds 35 as a minimum in the 
Horizontal Miss Distance | HMD* feet 4000 quantitative definition of 


well clear. Table 1 shows 
the numeric values used to 
define the DAA Well Clear 
threshold. 

The horizontal separation requirements are segregated from the vertical separation 
requirements. In other words, if an aircraft is adequately separated in the vertical plane to avoid 
a loss of DAA Well Clear, the horizontal separation has no effect, and vice versa. Figure 1 


Time to Co-Altitude Ty seconds 0 


shows a simplified notional depiction of the DAA Well Clear definition, though its actual shape is 
more complex. The figure also separates vertical and horizontal separation requirements. 

In both the horizontal and vertical dimensions, the definition incorporates a time and a 
distance constraint on the separation requirement. In the horizontal dimension, both time and 
distance constraints must be violated in order to have a loss of DAA Well Clear; whereas, in the 
vertical dimension, violation of either the time or distance constraints will result in a loss of DAA 
Well Clear. 

In the horizontal 
dimension, as shown in 
Figure 1, the aircraft T oq X RangeRate 
must remain outside of 
a time-based boundary -- 
(t.og) unless the ane” / \ 
projected minimum er \ 
distance between the rH of ==. ’ k ] 
two aircraft is greater Bo hice ti | 
than the specified Tee OY 7 
HMD*. The HMD is a ———- 
kinematic projection 
using the velocity 
vectors of the ownship 
and intruder aircraft. 
Thus, the maneuvering 
aircraft must turn to a 
heading thatachievesa | =~ TW | GURUTIUIDIUITITI pls File ae 
projected value of HMD ax +h, 
greater than HMD* to ...—_ k$ SSS eee is 
achieve well clear as Figure 1. Simplified Depiction of DAA Well Clear. 
opposed to _ being 
physically offset by a distance of HMD*. The angle that provides the required HMD projection 
varies greatly with the initial range between the encountering aircraft. 

In the vertical plane, as shown in Figure 1, the aircraft must remain outside of a time- 
based boundary (t;) and a spatial boundary defined by +h*. In contrast to the horizontal 
separation requirements, the vertical separation is not a projection but an absolute vertical 
distance between the aircraft. Since the vertical boundary is fixed, the time required to reach the 
specified vertical separation h*, or tg,, must be analyzed. [4] 


tan Tmoa x RangeRate 


2.2 Metrics 


2.2.1 Severity of Loss of Well Clear (SLoWC) 


The Severity of Loss of Well Clear (SLOWC) metric is used to assess the severity of 
Loss of DAA Well Clear on a per-encounter basis by capturing the most serious instance of 
Loss of Well Clear throughout an encounter. Based on the Well Clear definition, this severity 
metric is assessed based on the severity of the local penetration into all three of the Well Clear 
components: Horizontal Proximity, Horizontal Miss Distance Projection and Vertical Separation. 
Further information regarding the SLOWC metric can be found in the DAA MOPS Appendix 
L.5.1.5 [5]. 

The combined severity at any instance during an encounter from all three components 
can be expressed as: 


Equation 3: Calculation of SLoWC for Each Time Step 


SLoWC; = (1 — RangePen; ® HMDPen,; ® VertPen;) * 100% 


RangePen;, HMDPen,, and VertPen; are defined in the subsequent paragraphs. The 
Fernandez-Guasti Squircle operator, @, is used to combine the normalized penetrations from all 
three dimensions. 


Equation 4: Fernandez-Guasti Squircle Operator Definition. 
xOy B= Vx27+(1—-x2)y? 


The overall SLOWC penetration for the entire encounter is: 


Equation 5: Calculation of SLoWC for Encounter. 
SLOWC = MAX(SLoWC,) 


The resulting SLOWC ranges from 0% to 100% with 0% indicating Well Clear, and 100% 
representing full penetration into the Well Clear protection volume, i.e. both aircraft at the same 
place at the same time. 


2.2.1.1 SLOWC Horizontal Proximity 


The normalized horizontal proximity penetration is analogous to assessing the 
penetration into the t,,,q dimension. 


The local normalized horizontal proximity penetration is defined as: 


Equation 6: Calculation of RangePen. 


T; 
RangePen; = MIN (Z, 1) 
i 


Where the required horizontal range, Si, given the local horizontal range rate and Well 
Clear’s DMOD and Tou’ yields: 


Equation 7:Calculation of Required Horizontal Range, S. 


1 ° * o * 
S; =MAX (omop, alt (TiThog)? + 4DMOD2 — fiThoa)) 
where: 


i = [t1, t2,t3,°+ Cend—1 Cena] 
DMOD = 4000’ 

tno = 30S; 

r,; = horizontal range 

7, = horizontal range rate 


The resultant normalized horizontal penetration produces a value ranging from one to 
zero with one indicating the edge of t,,,4, and zero representing full penetration into the 
horizontal proximity dimension. 
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2.2.1.2 SLOWC Horizontal Miss Distance Projection 


The normalized HMD penetration is based on the ratio of the local HMD projection 
versus Well Clear’s DMOD requirement: 


Equation 8: Calculation of HMDPen. 


HMD; : 
DMOD’ 


| 2 
HMD, = (dx + V_xt cpa)? +(dy + Vyytcpa) 


HMDPen, = MIN ( 


L 


where: 

— Sex + ayy 
Vrx + Very 

HMDI,..4< q=% 


i = [t,, t2,t3,-+* tena—1 tenal 

dx = Aircraft separation in the x — direction using truth data 
dy = Aircraft separation in the y — direction using truth data 
Vy = Relative velocity in the x — direction using truth data 
Vyy = Relative velocity in the y — direction using truth data 


The resultant normalized HMD Penetration yields a value ranging from one to zero with 
One indicating the edge of DMOD, and zero representing full penetration into the DMOD 
requirement. 


2.2.1.3. SLOWC Vertical Separation 


The normalized penetration of the vertical component is assessed based on the ratio of 
local vertical separation, dh; versus Well Clear’s h*: 


Equation 9: Calculation of HMDPen. 


dh; 
VertPen; = MIN ( = ,1) 
where: 
= [bis tarts? tana tong | 
hx= 450’ 
dh; = abs(h1,i- h2,i) 


Similarly, the resultant normalized vertical penetration produces a value ranging from 
one to zero, with one indicating the edge of vertical threshold and zero representing full vertical 
penetration into the h*requirement. 


2.2.2 Alert Jitter 


Alerting is based on sensed relative position and velocity of intruder aircraft, per 
Appendix Q of the Phase 1 DAA MOPS [5]. Since alerting is a function of sensor-degraded 
data, it is therefore subject to uncertainty associated with sensor noise. A nuisance 


characteristic associated with alerting is an unnecessary number of transitions between alert 
levels. The resulting alerting performance may cause a decrease in safety due to pilot 
distraction or the pilot’s lack of trust in the alerting system. To measure the number of 
transitions, the metric Alert Jitter is defined. Alert Jitter is the number of increasing alert 
transitions which occurred throughout the encounter. An increasing alert transition is defined as 
a transition between no alert to any other alert level (preventive, corrective, or warning), as well 
as from a lower alert level (i.e. preventive) to a more severe alert level (i.e. corrective). Only the 
increasing alert levels are measured to minimize duplicate measurements. For example, for a 
simple co-altitude encounter the alert sequence is expected to be from <No Alert> to 
<Corrective Alert> to <Warning Alert>, which is represented by an Alert Jitter value of 2. 


3 Simulation Environment 


A fast-time simulation tool was developed to model UA system components including an 
aircraft performance model (2PAIRS), models of three sensor types, a DAA Tracker model, a 
deterministic pilot model, and the reference DAA algorithm Detect and AvolD Alerting Logic for 
Unmanned Systems (DAIDALUS [6])from the forthcoming Phase 1 DAA MOPS. The tool also 
provides for intruder states to be read in from a file or for intruders to fly straight and level 
trajectories during each run. One ownship and multiple intruders are supported by the tool but 
the results presented here are from a single intruder. There are three paths for data flow within 
the simulation tool: 

1. The first path, labelled “Truth” in Figure 2, provides true (i.e., perfect) state data as output by 
2PAIRS to DAIDALUS, which outputs guidance to the pilot model. The pilot model then 
determines a heading change command with respect to the truth-based guidance bands. 

2. The second path, labelled “Sensed” in Figure 2, provides true state data to the 
Sensor/Tracker suite which degrades the state data using sensor uncertainty assumptions. 
After degradation, the state data is provided to DAIDALUS which outputs guidance bands 
based on the degraded state data of the ownship and intruder. The pilot model then 
determines a heading change command with respect to the degraded state based guidance 
bands. 

3. The third path, labelled “Mitigated” in Figure 2, provides true state data to the 
Sensor/Tracker suite which degrades the state data using sensor uncertainty assumptions. 
After degradation, the state data is then provided to the SUM Mitigation software, which acts 
as a wrapper around DAIDALUS. The SUM Mitigation wrapper determines the positions and 


“Truth” 
>| DAIDALUS }|—» Pilot Model 
“Sensed” 
2PAIRS -— >| DAIDALUS }|—» Pilot Model 
zx 
! Sensor Tracker | 4 
Model Models | ; 
FAA Tech Center SUM Mitigation 


'_»| | DAIDALUS |}—») Pilot Model 


“Mitigated” 


Figure 2. Simulation Environment 


velocities of the ‘phantom’ intruder aircraft and provides the additional state data to 
DAIDALUS. DAIDALUS then outputs guidance bands based on the degraded state data of 
the ownship, the sensed intruder aircraft states, and the phantom aircraft states. The pilot 
model then determines a heading change command with respect to the resulting guidance 
bands. 
More detailed descriptions of each component of the simulation tool shown in Figure 2 
are provided in the following sections. 


3.1 Aircraft Dynamics Assumption 


A model of aircraft dynamics in a lateral maneuver was used. The simulation tool 2 
degrees-of-freedom Prototyping Aircraft Interaction Research Simulation (2PAIRS) [7] was used 
to model the UA’s dynamics throughout the encounter. When prompted by the pilot model, the 
aircraft initiates a 3 deg/sec turn at 5 deg/sec roll rate. The assumed maneuver rates and 
accelerations of this study did not stress the performance model; thus, the commanded 
maneuvers were achieved and maintained. 

2PAIRS accepts multiple input formats depending on the particular study. For ownship, 
2PAIRS accepts first-order aircraft parameter values (e.g., maximum Thrust-to-Weight, wing 
loading, coefficient of lift) and is able to define the ownship’s trajectory assuming level, 
unaccelerated flight. Alternatively, 2PAIRS accepts ownship trajectory files as input. The input 
trajectory is followed until the DAA system and pilot model determine that a maneuver is 
required, at which point the 2PAIRS flight dynamics assumptions over-ride the input trajectory 
array and a maneuver is performed. When provided an array, 2PAIRS estimates the attitude 
and state information of the UAS. 

For intruder aircraft, 2PAIRS accepts 3-dimensional trajectory arrays as input. The 
intruder’s positon and velocity follow the input arrays; aircraft dynamics are not calculated for 
the intruder aircraft. 


3.2 Pilot Model 


A simplified pilot response model was implemented to minimize the variability in the 
response caused by maneuver selection and timing. The pilot model commanded a lateral 
maneuver in the positive heading direction (i.e., turn right). 

The timing of the maneuver was constant relative to an estimated Time to Violation 
(TTV) output by the DAIDALUS algorithm. The maneuver was commanded when TTV = 10 
seconds, or 10 seconds prior to a DAA Well Clear volume penetration. Using a constant TTV 
provides a consistent time of response relative to the airborne hazard. Per MOPS specifications, 
an alert should be issued no less than 15 seconds prior to a LOWC (TTV = 15 seconds). 
Allowing 5 seconds for the pilot to input the commanded maneuver and for the maneuver to 
reach the airborne UA results in the choice to maneuver at TTV =10 seconds. 


3.3 Sensor and Tracker Model 


Simulation models representing three types of airborne surveillance sensors were used: 
Automatic Dependent Surveillance — Broadcast (ADS-B), Active Surveillance Transponder 
(AST), and Air-to-Air RADAR. The tracker model is a best source selection tracker. To reduce 
the impact of the tracker model’s best source criteria, only single sensor data is passed to the 
tracker. The simulation models representing the three types of sensors were developed by the 
FAA’s William J. Hughes Technical Center in support of SC-228 modeling and simulation 
efforts. Further information on the Tracker model and Sensor models can be found in Phase 1 
DAA MOPS Appendix F and Appendix Q [5], respectively. 


3.3.1 RADAR 


The radar model embodies the error among readings of range, range rate, azimuth, and 
elevation. The directional nature of the radar sensor necessitates the modeling of both a 
detection range and a field of regard. The table below holds values per state describing the 
modeled amount of error per sigma and the bias associated with each distribution as well as the 
update rate, detection range, and field of regard state restrictions. 

Runs incorporating only radar degraded data are representative of flight operations 
involving an equipped UAS and a non-cooperative intruder. Non-cooperative intruders are 
aircraft operating in the NAS with no transponder (i.e., aircraft not broadcasting their location to 
other NAS users via ADS-B or responding to transponder interrogations from Active 


Surveillance systems). 


Bearing 


Table 2. Radar Model Parameters. 


Relative Error (1-sig) 
15.24m (50 feet) 
3.0m/s (10 feet/second) 


15.24m (50 feet) 
2.4m/s (8 feet/second) 


Altitude 


1-degree 0.5-degree 


Update Rate 1 second 
Tracking Range 5 nmi (< 100kts. Intruder) Detection Range Scale 
6.5 nmi (100-300kts. Factor 
Intruder) Az: [0, 30], 1.0 
8 nmi (> 130kts. Intruder) Az: [30, 60], 0.84 
Az: [60, 90], 0.46 
Az: [90, 110], 0.45 


Field of Regard 


+/- 15 Elevation (Stabilized with respect to velocity vector) 
+/- 10 Azimuth 


Probability of Track 


3.3.2 Active Surveillance 


Pr(Track) = 1 


Active surveillance uses the standard TCAS transponder interrogation that provides 
range, bearing, and altitude to the intruder. The following table provides the assumed model 
performance for both Mode-S and Mode-C active surveillance. 
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AST-only runs are representative of flight operations involving a properly-equipped UAS 
and an intruder equipped with either a Mode C or Mode S transponder. Mode A transponders 
were not considered because such systems are rare. 


Table 3. Active Surveillance Model Parameters 


Relative Error (1-sig) Quantization 
Range 15.24m (50 feet) 38.1m | --- 
(125 
feet) 
Bearing [-10, 10 degrees]: 9 degrees RMS; --- --- 


maximum 27 degrees 


[-15, -10] or [10, 20 degrees]: 15 
degrees RMS; maximum 45 degrees 


Altitude 0 Per Quantization 25 
TSAA | ft. / 100 ft. 

Model | (Intruder Aircraft) 
/ 1 ft. (Qwnship 
Aircraft) 


State WEEVLO(=) 


Update Rate 1 second 


Detection Range | Mode C =< [9.5, 6, 3.3] nmi 
Mode S = < 15.6 nmi 


Probability of Mode C = 0.90 
Reception/Detecti | Mode S = 0.95 
on 


Field of Regard [-15, +20 degrees) Elevation 


3.3.3 ADS-B 


Automatic Dependent Surveillance—Broadcast (ADS-B) is a precise satellite-based 
surveillance system. ADS-B Out uses GPS technology to determine an aircraft's location, 
airspeed and other data, and broadcasts that information to a network of users, including other 
airborne aircraft. Although ADS-B has a variety of performance levels, the following 
assumptions were used to model ADS-B equipped intruder aircraft. 


Runs incorporating ADS-B only sensor degraded state data are representative of flight 
operations involving intruder aircraft equipped with ADS-B Out. 


Table 4, ADS-B Model Parameters. 


Absolute Error Bias Time 
(per AC) 1-sig Correlation 
NACp = 7 Horizontal 75.6m 0 300 sec 
Position 
Baro Altitude 0 Per TSAA 
model 
NACv = 1 Horizontal 4m/s 0 300 sec 
Velocity 
Vertical 1.707m/s (95%) 
Velocity 
State | WENT 
Update Rate dt = 1 second 
Latency Effects <.4sec 
(Uncompensated) 


Detection Range DR = <20NM 


Probability of PD: 0.95 
Reception / 
Detection 


3.3.4 Ownship Navigation (NAV) Model 


The ownship navigation system is based on Inertial Navigation System / Global 
Positioning System (INS-GPS) performance, and provides the ownship’s position, velocity, and 
attitude information. The following table provides the assumed performance of the ownship 
navigation system, which in large part is based on expected ADS-B performance. 

For attitude estimation, the tracker is provided true attitude state information (roll, pitch, 
yaw) of the ownship. The attitude is then degraded using the attitude absolute error standard 
deviations shown in Table 5. 
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Table 5. Navigation (NAV) Model Parameters. 


Absolute Error Time 
(per AC) 1-sig Correlation 
Horizontal | NACp = 7 75.6m 0 300 sec 
Position 
Baro Altitude 0 ICAO 10 
Annex Bias 
model 
Horizontal | NACv=1 Am/s 0 300 sec 
Velocity 
Vertical Velocity 1.707m/s (95%) 
Attitude [0.2, 0.2, 0.4] degs 
[Roll, Pitch, Yaw/Heading] 


3.3.5 Tracker 


The primary function of the Tracker is to receive surveillance data from various sensors, 
form a single track for each detected target, and present the track information together with 
ownship information to the DAA algorithm for use in alerting and guidance functions. The 
Tracker model utilized best source selection criteria to provide a single track to the DAA system. 
In order to avoid complications caused by source selection, a single surveillance source was 
used at a time. 


3.4 DAIDALUS 


DAIDALUS is a software implementation intended to satisfy the operational and 
functional requirements detailed in NASA’s DAA concept of integration for UAS [2]. In particular, 
DAIDALUS provides algorithms that: 1) determine the current, pairwise well-clear status of the 
ownship and all aircraft inside its surveillance range, 2) compute maneuver guidance in the form 
of ranges of maneuvers that a pilot-in-command (PIC) may take that will cause the aircraft to 
maintain or increase separation from the well clear violation volume, or allow for recovery from 
loss of separation in a timely manner within the performance limits of the ownship aircraft, and 
3) determine the corresponding alert type, based on a given alerting schema, corresponding to 
the level of threat to the well-clear volume. In the developed simulation tool, DAIDALUS utilized 
a thresholds-based alerting schema [6]. 


4 Sensor Uncertainty Mitigation (SUM) 


Sensor uncertainty will cause Losses of DAA Well Clear (LoWC) frequently unless the 
uncertainty in position and velocity is accounted for. This section describes the Sensor 
Uncertainty Mitigation (SUM) approach used for the current study and how the scaling factors 


were tuned. 


4.1 SUM Approach Description 


In the north and east dimensions, the north, east, and north-east (covariance) standard 
deviation estimates are analyzed using eigenvalue/eigenvector decomposition to determine the 
major and minor error ellipse axes and magnitudes. The angle of the major/minor axes of the 
uncertainty ellipse is determined by the ratio of largest eigenvector. This is done because radar 
and AST sensors typically have much better accuracy in range than in azimuth, which makes 
the error ellipse highly elongated. Generally, the major and minor horizontal velocity error ellipse 
axes are closely aligned with those of the positional error, and for this mitigation algorithm they 


are assumed to be coincident. 


Table 6. Sensor Uncertainty Mitigation 
scaling factors 


Scaling Factor Numeric Value 


Horizontal 
Position 1.5 
Horizontal 
Velocity a 


Vertical Position 1.0 


Vertical Velocity | 1.0 


For generating intruder input to 
the DAA algorithm, each intruder's 
sensed state (3D position and velocity) 
is passed to the DAA algorithm along 
with a set of ‘phantom’ intruder states. 
These phantoms are generated by 
enumerating displacements of the 
sensed state (position and velocity) of 
the intruder in the positive and negative 
direction of each of the horizontal error 
ellipse axes and the vertical axis using a 
scaling factor multiplied by the matching 
standard deviation for that axis. This 
effectively puts an intruder at the 
vertices of a box bounding the error 
ellipses corresponding to the scale 
factors. The numeric values of the 
scaling factors, presented in Table 6, 
were chosen based on a study detailed 
in Section 4.2. 


Sensed position 
and velocity 


Figure 3. Notional Depiction of Sensor Uncertainty Mitigation (SUM) 
Volume. 


In performing the enumeration, an additional set of phantom intruders is placed either at 
the sensed intruder altitude and altitude rate, or, if the sensed altitude of the intruder could be 
coincident with ownship's altitude within the look ahead time, at the ownship altitude and vertical 
rate (zero vertical displacement). This middle plane ensures that potential conflicts are not 
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missed by propagating ownship between the upper and lower surfaces, which in the case of 
radar can be widely spaced due to the large vertical uncertainty. 

As a final processing step, the altitude and speed bands of the DAA algorithm are post- 
processed to remove and non-conflict regions that lie entirely within the upper and lower 
surfaces (as propagated out to the look ahead time) or that represent ownship vertical speeds 
that would not escape the upper or lower surface within the look ahead time. 


4.2 Tuning of SUM Scaling Factors 


The position and velocity of the SUM ‘phantom’ intruders are dependent on the scalar 
multiplication of the uncertainty estimate provided by the DAA Tracker. Determining the value of 
the scalar multiple used in the SUM approach is a function of the UA system, DWC definition, 
alerting times, and other parameters. To optimize the performance for the MOPS representative 
DAA system, a multiplier tuning study was done. This section outlines the study as well as the 
methodology used to determine the optimal SUM Scaling Factor settings. The robustness of the 
tuning was not evaluated. Similar methodology can be used to tune the SUM approach for 
differing MOPS, DWC, and Alerting Criteria. 


4.2.1 Experiment Design 


Simulation runs were completed for every combination of encounter and SUM scaling 
factors in the test matrix as detailed in the following sections. Due to the stochastic nature of the 
sensor-tracker models, replicates were used. For each combination of encounter, SUM scaling 
factors, and sensor type, the following number of 


replicates were used for each guidance type: 1 Table 7. Experiment Design - Encounter 
Truth replicate, 10 Sensed replicates, and 10 Parameters 
Mitigated replicates. 

The complete tuning experiment design Factor Value 
consists of 612 total encounters. With the UA Airspeed 130 KTAS 
specified number of replicates, the resulting Intruder Airspeed 100, 200 KTAS 
experiment contained 32,436 total runs. | Relative Bearing | -45,0,45deg | 

UA Turn Rate 1.5, 3 deg/sec 


4.2.2 Encounter Set 


A full factorial of the encounter parameters in Table 7 was used to define the encounter 
set used to identify the SUM Scaling Factor setting which provided the best response. The 
resulting encounter set contains 36 unique encounters. 


Figure 4 depicts the 
geometries of the encounters 
used in the tuning study. The 
ownship, shown in blue, is 
flying straight and level at a 
constant 130 KTAS, and 
maneuvers to the right when 
prompted by the pilot model. 
The intruder aircraft are 
positioned such that there will 
be a direct collision at t = 120 
seconds if no avoidance 


maneuver is performed by the @ Ownship Airspeed = 130 KTAS 
UA. The non-maneuvering der Ai OD togcik 

intruders are flying at airspeeds @ Intruder Speed = 00 KTAS 
chosen such that there are © Intruder Airspeed = 200 KTAS 


intruder aircraft flying both 
slower and faster than the UA. 
Note the change in relative velocity vectors between the ownship and the two intruder airspeeds 
at +- 45 deg. 

No encounters were evaluted with the UA overtaking the intruder because they are not 
stressing cases. 


Figure 4.Tuning Study - Encounter Geometry. 


4.2.3 Scaling Factors 
; : Table 8. SUM Scaling Factors experiment values. 
A 2-level full factorial design was 


used to analyze the response with the four Factor Min. Max. Center 
SUM scaling factors being the independent Value Value Value 
variables. Table 8 contains information on Scale XY 0.5 1.5 1.0 
the factor levels. Including a single center Scale “Vxy 0.5 15 1.0 
point, the 2-level factorial design resulted Scale Z 05 15 1.0 

in 17 unique combinations of SUM Scaling Scale Vz 05 15 1.0 
Factors. a a a 


4.2.4 Tuning Methodology 


Using SLOWC as the primary response of interest and Alert Jitter as a secondary 
consideration, the objective of this tuning study was to minimize both measured responses 
during the specified encounters. Each combination of SUM Scaling Factors in the 2-level 
factorial was analyzed independently. After collecting data from the encounter set described in 
Section 4.2.2, each response was compared. Choosing the scaling factor combinations that 
minimize SLOWC are first identified, then those combinations’ alert jitter responses were 
analyzed. 

The focus of this report is analysis of the SUM approach so the details of selecting the 
multiples used is not provided. The multipliers used are given in Table 6. 


4.3 Guidance Band Buffer Analysis 


The uncertainty associated with sensors Is a source of alert jitter. Per the Phase 1 DAA 
MOPS [5], Corrective and Warning alerts are issued when a hazard is predicted to be on the 
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UA’s current trajectory. If the UA attempts to fly tangential to the hazard volume, sensor noise 
will likely cause a large increase in the Alert Jitter. Such a tangential flight path is associated 
with following the edge of the DAIDALUS guidance bands. To minimize the amount of Alert 
Jitter while also keeping the path deviation reasonably constrained, a trade-off study was 
conducted. The study measured the impact of adding a buffer to the edge of the DAIDALUS 
guidance bands on SLOWC and Alert Jitter. A constant buffer of 1 deg, 5 deg, and 10 deg was 
added to the edge of the guidance band and analysis was done to assess alert jitter for each. 


4.3.1 Encounter Set 


This study was performed using a full factorial of 6 factors, as detailed in Table 9. The 
encounter geometries are presented in Figure 5. The resulting experiment matrix consists of 54 
unique encounters. To account of the stochastic response from the sensor/tracker models, 
several replicates were run for each encounter. A total of 21 replicates were used for each 
encounter: 1 Truth replicate, 10 Sensed replicate, and 10 Mitigated replicate. The resulting 
experiment matrix consist of 1134 runs for each Guidance Band Buffer value. 


Table 9. Factor Values for Guidance Band Buffer Analysis 


Parameter (Units) | Values 

UA Turn Rate 0 (open-loop), 1.5, 3.0 

(deg/sec) 

UA Airspeed 130 

(KTAS) Figure 5. Encounter Geometry for 
Intruder Airspeed 100, 200 Guidance Band Buffer Analysis. 
(KTAS) 

Relative Bearing 0, +45 

(deg) 

Sensor RADAR, AST, ADS-B 

Configuration 


4.3.2 Results 


The results presented herein are separated based on the pilot model's ability to respond. 
In other words, analysis of Open-Loop runs is considered independently from Closed-Loop 
analysis. 


4.3.2.1 Open-Loop Alert Jitter 


Simulation runs in which the pilot response model does not follow the suggested 
guidance and thus maintains the initially defined trajectory are referred to as Open-Loop 
simulation runs. Such runs are useful for analysis as they remove the influence of the pilot 
response model from the measure responses of interest. Alert Jitter response is of particularly 
interest for Open-Loop analysis, as Open-Loop runs remove influence of maneuver selection 
and Guidance Band Buffer value. For the defined encounter set, all Open-Loop runs result in a 
direct collision with a SLOWC = 100%. Figure 6 through Figure 8 shows the normalized 
distribution of Alert Jitter for the open-loop runs of the previously described encounter set. The 
figure shows the normalized distributions of Alert Jitter grouped by path of data flow, as detailed 
in Figure 2. Within Figure 6 through Figure 8, the histogram of runs using truth data are shown 
in blue, the runs using sensor/tracker degraded data are in the red, and runs which incorporate 
the SUM approach are in green. 

As expected with perfect surveillance data, the alert jitter equals 2 for all encounters. 
Given that the simple, benign encounter set involves co-altitude encounters, the ideal alert 
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Figure 6. Distribution of Alert Jitter: Open Loop - RADAR only 


transition is from <No Alert> to <Corrective Alert> to <Warning Alert> which is represented by 
an Alert Jitter value of 2. Note that there is no difference between the Truth runs with different 
sensors, since the limitations of each sensor are not incorporated in the Truth runs. 

Figure 6 shows the Alert Jitter distribution for encounters involving a non-cooperative 
intruder sensed by the air-to-air radar. Comparing the Truth and Sensed distributions, the 
degradation of state data undesirably increases the Alert Jitter. 
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Figure 7. Distribution of Alert Jitter: Open Loop — AST only 


Figure 7 shows the Alert Jitter distribution when equipped with AST. There is no 
significant improvement in Alert Jitter performance when comparing Sensed and Mitigated runs. 
The maximum Alert Jitter is 7. Of interest, there are occurrences of Alert Jitter equal to 1 when 
equipped with AST indicating that an alert was missed (e.g., UA transitioned from <No Alert> to 
<Warning Alert> without transitioning to <Corrective Alert>). 

ADS-B-equipped Alert Jitter distribution is shown in Figure 8. As ADS-B is the most 
accurate sensor, the large majority of the runs resulted in Alert Jitter of 2. Even when following 
guidance based solely on ADS-B degraded data, the Alert Jitter performance is well contained. 
Even then, SUM approach increases the percentage of runs with Alert Jitter equal to 2. 
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Figure 8. Distribution of Alert Jitter: Open Loop - ADS-B only 


4.3.2.2 Closed-Loop SLoWC 


SLOWC response is analyzed for closed loop runs to investigate the severity of the loss 
of separation with the pilot in the loop responding to DAA guidance. 

Figure 9 — Figure 11 show sensor-specific distributions of SLOWC for the three Guidance 
Band Buffer increments. Each figure contains three sub-figures with increasing Guidance Band 


Buffer: 1 degree Guidance Band Buffer at the top, 5 deg in the center, and 10 deg at the bottom. 
Each sub-figure contains 3 distributions with different colors: Blue — Truth, Red — Sensed, 
Green — Mitigated. In all figures, the runs with guidance based on truth data maintain DAA Well 
Clear (SLOWC = 0) in all encounters. Table 10 - Table 12 contain summary statistics with the 


maximum SLOWC and Alert Jitter from each sensor-specific distribution. 


RADAR-only distributions of SLOWC are shown 
in Figure 9. Independent of Guidance Band Buffer 


Table 10.Maximum SLOWC - RADAR. 


increment, the SUM mitigation approach reduces the Guidance | Guidance | Max. Max. 
measured SLOWC compared to following guidance Band Type SLOWC | Alert 
based only on sensor degraded state data. There Buffer Jitter 
appears to be an increase in SLOWC values for runs (deg) 

with Guidance Band Increment greater than 1. While 1 Truth 0 2 
the figure indicates runs with a SLOWC ranging from 1 Sensed 84.67 8 
greater than 0 to less than 10, the maximum SLOWC 1 Mitigated 0 8 
experienced for runs incorporating the SUM approach 5 Toth 0 5) 
is 0.85% with 5 deg buffer and 6.37% with a 10 deg 5 Sencead 18.06 9 
buffer, as shown in Table 10. These values of SLoWC 5 Mitigated 0 35 6 
are very small. Analyzing the maximum Alert Jitter 10 Truth 0 5 
response for RADAR-only runs with the SUM approach 

incorporated reveals a decrease in the Alert Jitter as 10 Sensed 5.65 9 
Guidance Band Buffer increases. 10 Mitigated | __6.37 2 


Figure 10 shows the AST-only distributions of 
SLOWC. While SLOWC is reduced using the SUM approach, the performance is still not ideal. 
From Table 11, the Mitigated runs result in a SLOWC greater than 56% regardless of Guidance 
Band Buffer value. This performance is likely to be consider unacceptable. For AST, SUM 
approach undesirably increases the Alert Jitter measured. This is likely due to the large 


variations that occur in AST measurements. 


ADS-B only distributions of SLOWC are shown in Figure 11 and the maximum value of 
SLOWC is shown in Table 12. There were minor loses of DAA Well Clear for all levels of the 
Guidance Band Buffer with slight improvement as the Guidance Band Buffer increases. Alert 
Jitter improves greatly for Guidance Band Buffer greater than 1 deg, but there is little difference 
between 5 deg and 10 deg buffers. 

Based on this discussion, future use of this simulation tool will employ a 5 deg band 
buffer to optimize the trade-off between minimizing SLOWC and Alert Jitter while also minimizing 
the required avoidance maneuver. 


Table 11.Maximum SLOWC - AST. 


Table 12.Maximum SLoWC —- ADS-B. 


Guidance | Guidance | Max. Max. Guidance | Guidance | Max. Max. 
Band Type SLOWC | Alert Band Type SLOWC | Alert 
Buffer Jitter Buffer Jitter 
(deg) (deg) 
1 Truth 0 2 1 Truth 0 2 
1 Sensed 96.43 12 1 Sensed 12.52 9 
1 Mitigated 65.87 20 1 Mitigated 2.62 14 
5 Truth 0 Z 5 Truth 0 2 
5 Sensed 97.45 11 5 Sensed 10.41 8 
5 Mitigated 56.28 17 5 Mitigated 1.95 8 
10 Truth 0 2 10 Truth 0 2 
10 Sensed 98.70 13 10 Sensed 7.79 7 
10 Mitigated 76.54 18 10 Mitigated 1.07 9 
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Figure 9. Distribution of SLoWC: Closed Loop - RADAR equipped. 
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Figure 10. Distribution of SLoWC: Closed Loop -AST equipped. 
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Figure 11. Distribution of SLoWC: Closed Loop - ADS-B equipped. 


5 Discussion and Analysis 


The introduction of additional ‘phantom’ intruders augments the DAA region by creating 
a physically larger volume for the DAA system to avoid as well as augmenting the perceived 
time at which the maneuver must begin. The following sections detail the magnitude of the 
change to the DAA avoidance region by analyzing the change in commanded maneuver timing 
as well as the change in the provided guidance bands. 


5.1 Size of SUM Volume 


The additional ‘phantom’ intruders introduced by the SUM approach expand the hazard 
avoidance volume. Figure 12 — Figure 14 depicts a pairwise head-on encounter with a 
maneuvering ownship. Within each figure, the true positions of the ownship (blue) and intruder 
(green) are shown as well as each sensor degraded track (black). The ‘phantom’ intruder 
positions (red) are also shown. Each figure shows the overhead view on the left and a profile 
view on the right. For each encounter shown, there would be a direct head-on collision between 
the two aircraft if no avoidance maneuver is performed. 

Figure 12 shows the same pairwise encounter involving a non-cooperative intruder 
sensed only by the onboard air-to-air RADAR. The uncertainty estimate starts out very large but 
improves as the range between the aircraft decreases, resulting in a reduction in the size of the 
volume defined by the SUM phantom intruders. 
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Figure 12. Example of Sensed Position - RADAR. 


The same pair-wise encounter involving AST equipped aircraft is shown in Figure 13. 
The sensed position of the intruder is particularly poor in the lateral positioning. Initially, the 
sensed position of the intruder (shown in black) is more than 1 nmi off from the true position. 
With the SUM approach a ‘phantom’ intruder is roughly 3 nmi off laterally offset. This collision 
avoidance region is very large and may inhibit lateral maneuvers. 
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Figure 13. Example of Sensed Position - AST. 


Figure 14 shows the pairwise encounter with ADS-B as the equipped sensor. As ADS-B 
is not a relative sensor, but rather a received message with high accuracy comparable to the 
own ship’s sensed position accuracy, the estimated uncertainty is relatively small and nearly 
constant, shown in Figure 14 by the smoothness of the sensed intruder track (black). The figure 
on the right shows a constant vertical bias of ~250 ft. 


ADS-B - Mitigated 104 ADS-B - Mitigated 
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Figure 14. Example of Sensed Position - ADS-B. 


5.2 Effect on Lateral Maneuvering 


To illustrate the SUM approach’s effect on maneuvering and the resulting SLOWC, a 
case study is provided. Figure 15 — Figure 17 show the maneuver performed in response to 
DAA guidance based on the specified sensor configuration. Each figure shows data for the 
same encounter scenario; a pair-wise head-on encounter involving a maneuver ownship at 130 
KTAS and a constant trajectory intruder at 100 KTAS. The UAS is capable of turning at 3.0 
deg/sec. Each figure contains two subplots; on the left is the overhead view of the encounter, on 
the right is normalized distribution of SLoWC measured for the depicted simulation runs. In each 
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Figure 15. Effect of Mitigation - Maneuver Selection - RADAR. 


figure, the black represents the runs following guidance based on perfect state data (i.e., Truth), 
the blue depicts the runs in which the maneuver is based on sensor degraded state data only 
(i.e., Sensed), and the red line represents runs incorporating the SUM approach (i.e., Mitigated). 

Figure 15 shows the effect on maneuver selection for encounters involving non- 
cooperative intruders sensed solely by the air-to-air radar onboard the UA. The maneuver 
selection is well-contained with respect to the source of the provided guidance. In the trajectory 
figure, there are tight groupings of the Sensed and Mitigated runs indicating a consistent 
uncertainty estimate across the replicates. The Mitigated runs make a large path deviation and 
turn further than the Sensed runs. From the SLoWC distribution of Figure 15, all Truth runs and 
the majority of Mitigated runs resulted in no LOWC, while most Sensed runs failed to maintain 
DAA Well Clear though none exceeded 10% SLOWC. 

The effect on maneuver selection for encounters involving transponder equipped aircraft 
is shown in Figure 16. In this example encounter, the UA displays insufficient ability to maintain 
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Figure 16. Effect of Mitigation - Maneuver Selection - AST. 
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DAA Well Clear. The variation in AST estimates of intruder state data and associated 
uncertainty are too great to maintain separation from other aircraft. Despite the poor 
performance when utilizing AST sensor degraded data, the normalized distribution of SLowC 
shows a reduction in SLOWC for Mitigated runs compared to Sensed runs. 64% of the Sensed 
encounters had SLOWC greater than 10% while only 26% of the mitigated ones did: a 40% 
improvement. 

Figure 17 shows the effect on maneuver selection for encounters involving ADS-B 
equipped aircraft. The trajectory subplot shows that the trajectories for all runs are closely 
bound. The runs incorporating the SUM approach are shown to maintain further separation from 
the intruder than the runs using sensor-degraded state data only. This is confirmed in the 
normalized distribution of SLOWC as most runs following the SUM approach-based guidance 
resulted in fewer losses of DAA Well Clear (i.e., SLOWC > 0). 
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Figure 17. Effect of Mitigation - Maneuver Selection - ADS-B. 


Conclusion 


Sensor uncertainty negatively impacts an Unmanned Aircraft System’s (UAS’s) ability to 
maintain sufficient separation from intruder aircraft and limit unnecessary alerting. An approach 
to compensate for sensor uncertainty for use in Detect-and-Avoid (DAA) systems was 
developed and presented. A methodology used to tune the SUM approach to a specific 
application was also introduced. Such methodology can be applied to a different system to meet 
the needs and operational considerations of a wide variety of UA. The SUM approach is 
dependent on the estimated uncertainty provided by the DAA Tracker, and thus is independent 
of sensor type. This allows the SUM approach to be applicable across a large number of UAS 
component configurations. 
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